NEPC 



NATIONAL EDUCATION 
POLICY CENTER 



Review oe Great Teachers 
AND Great Leaders 



Reviewed By 
Paul Shaker 

Simon Fraser University 
October 2010 



Summary of Review 

“Great Teachers and Great Leaders” (GTGL) is one of six research summaries issued by the U.S. 
Department of Education in support of its Blueprint for Reform. This review examines the presentation of 
research about improving teacher and administrator quality in GTGL. The review concludes that there are 
serious flaws in the research summary. The report, however, lacks sufficient analytic depth, does not 
present its evidence in a logical manner, makes sweeping claims, and draws conclusions based on weak 
data. . 




National Education Policy Center 



School of Education, University of Colorado 
Boulder, CO 80309-0249 
Telephone: 303-735-5290 
Fax: 303-492-7090 

Email: NEPC@colorado.edu 
http://nepc.colorado.edu 



Kevin Weiner 

Editor 

Alex Molnar 

Publishing Director 

Erik Gunn 

Managing Editor 

This is one of a series of Think Tank Reviews made possible in part by funding from the Great Lakes 
Center for Education Research and Practice, and it is also available at http://www.greatlakescenter.org. 



Suggested Citation: 

Shaker, P. (2010). Review of “Great Teachers and Great Leaders.” Boulder, CO: National Education 
Policy Center. Retrieved [date] from http://nepc.colorado.edu/publication/ great- teachers. 




Review of Great Teachers and Great Leaders 

Paul Shaker, Simon Fraser University 



I. Introduction 

In March 20 10, the Obama administration released a Blueprint outlining its proposals for 
reauthorizing the Elementary and Secondary Education Act (ESEA) . ‘ In May 20 10 the U. S. 
Department of Education (USDOE) followed with a set of six documents offered as “research 
summaries” supporting the administration’s plans.^ 

The second of these six reports is titled “Great Teachers and Great Leaders” (GTGL). It is 
divided into three sections: “Effective Teachers and Leaders,” “Teacher and Leader Pathways,” 
and “Teacher and Leader Innovation Eund.” The summary includes bibliographic references to 
approximately 80 sources and has five sidebars to illustrate case studies and examples. 3 

The content and form of the research summary departs from the usual standards for social 
science research syntheses. It distorts evidence by showcasing articles from the popular press, 
government publications, and advocacy think tank reports vrinle ignoring a great deal of 
relevant peer-reviewed scholarship. 



II. Findings and Conclusions of the Report 

“Great Teachers and Great Leaders” begins by setting out three claims: (a) “the interaction 
between teacher and student is the primary determinant of student success”; (b) “[r]esearch 
shows that top- performing teachers can make a dramatic difference in the achievement of their 
students”; and (c) it is very important for teachers and principals to access “meaningful 
information about their practice and [to] support them in using this information” (p. i). 
Building on these basic principles, the research summary then spells out several specific 
evidentiary claims and policy proposals. 

The first section of the GTGL research summary is called “Effective Teachers and Leaders.” It 
emphasizes that teachers and principals are “the key” to student success and points out that 
educators’ pay is generally not based on performance, but on years of service and courses taken. 
We do this, according to the report, despite there being “no evidence” (with the exception of 
mathematics teachers with content degrees) “that teachers with master’s degrees perform 
better” (p. 6). Further, it argues that professional development does not compensate for the 
purported ineffectiveness of graduate study, since teachers have insufficient professional 
development opportunities. It also argues that the guality of professional development is 
lacking. 
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Effective principals and teachers, the report argues, should be as available to high- poverty- 
schools as they are to other schools. Yet, identifying effectiveness is problematic. The report 
confidently asserts that “teacher qualifications” are not a predictor of student learning despite 



Generally, GTGL reeommends performanee pay based on outeomes more 
broadly defined than in the eurrent ESEA. 



ackno-wledging in the summary that “Most of the current teacher evaluation systems that are 
used fail to provide feedback [to help] differentiate effective from ineffective teachers” (p. 5). 

The summary also daims that an area of particular deficiency is teacher access to formative data 
that would improve instruction, gi-ving teachers timely information helpful for adjusting 
teaching. 

The first section condudes by noting that most compensation systems do not pay incentives for 
teachers to seek employment in challenging schools or in shortage areas such as math and 
science and that, consequently, “high-poverty and high- minority schools are least likely to have 
qualified and effective teachers” (p. 7). This contention is also made concerning the distribution 
of highly rated prindpals and other school leaders. 

Generally, GTGL recommends performance pay based on outcomes more broadly defined than 
in the current ESEA. It argues for implementing incentives for professional development and for 
creating a school en-vironment more flush with data that, in turn, would drive dedsion- making 
about this and other school processes. GTGL also strives for greater equity in school staffing to 
counter the tendency for outstanding teachers to avoid schools impacted by poverty. 

The second section of the research summary, “Teacher and Leader Pathways,” begins with two 
key assertions: (a) that “[mjany teacher preparation programs— traditional and alternative 
routes— are not preparing educators to succeed in today’s classrooms,” and (b) that “[mjany 
teacher preparation programs are not highly selective and do not set high standards for 
completion” (p. 17). Alternative programs are singled out for a lack of selecti-vdty. The summary 
also contends that there are promising models in operation, particularly ones that emphasize 
practice and reflection as well as performance or portfolio- capstone acti-vities. 

The report argues that improved prindpal preparation should focus on leading turnaround 
efforts and should emphasize instruction and leadership standards. Recruitment should be 
“selective and purposeful” (p. 18 ). Clinical experiences and school- based internships should be 
central to these programs. 

The third section, “Teacher and Leader Innovation Fund,” calls for the development of 
differentiated- compensation systems for educators based on existing exemplars. It goes on to 
critique districts that mismanage hiring and other human- capital- management systems and to 
argue that tenure and licensure do not take performance into account often enough. 
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III. The Report’s Rationale for Its Findings and Conclusions 



The stated rationale for GTGL and its companion documents is “to inform conversations around 
ESEA reauthorization.... These documents outline the research base around each section of the 
blueprint. ’’"‘The research summary includes many conclusions that are said to be drawn from 
the research literature, and the claim that GTGL “outlines the research base” around teacher and 
principal education invites a high standard of scrutiny. 



IV. Review of the Validity of the Findings and Conclusions 

The task of assessing the rigor of the research smnmaiy reguires consideration of two central 
issues: the guality of the dted research and the connection of that research to the conclusions 
drawn and policies recommended in the Blueprint. The first issue, concerning the research base, 
is addressed in the next two sections of this review. The second issue, discussed here, is one of 
the logic of GTGL’s arguments. 

The validity of GTGL hinges on its premises’ quality and their use in argument. Eor example, 
GTGL claims, “Of all the work that occurs at every level of our education system, the interaction 
between teacher and student is the primary determinant of student success” (p. 1). Shortly 
thereafter, it asserts, “The key to student success is providing an effective teacher in every 
classroom” (p. 3) . Yet, while it is arguably true that, of school factors, teacher quality is the 
primary determinant of success, it does not follow that school factors are the factors most vital 

When it comes to high-stakes judgments of teacher quality, the research 
summary implies by its omission that out-of-school factors are 
extraneous. The research base tells us otherwise... 

in ensuring or limiting student success. The problem here is that the research summary and the 
Blueprint proposals relating to teacher quality demonstrate a point of view that ignores out-of- 
school factors. GTGL is, in fact, silent on this issue. And since the report proposes tying 
measures of teacher quality to student outcomes, this oversight is extremely important. 

It should be acknovdedged that the report shows deep concern for students in high- poverty and 
high- minority schools. Moreover, the larger 5/ueprmt includes support for “Promise 
Neighborhoods” modeled on the Harlem Children’s Zone, which does take into accoimt out-of- 
school factors. But when it comes to high- stakes judgments of teacher quality, the research 
summary implies by its omission that out- of- school factors are extraneous. The research base 
tells us otherwise,^ and the fact that these out-of-school issues are considered elsewhere in the 
Blueprint does nothing to help here. The data regarding teacher effectiveness— a pillar of 
GTGL— are misleading if not examined in light of variations due to social context. For instance, 
one cannot assume that what makes for an effective teacher in a privileged setting makes for one 
in more challenging settings. 
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Of course better teachers are a boon to students in any situation. Where the research summary 
falls short is in not adequately supporting its more ambitious claim that teachers are the “key” to 
student success. The scant research offered includes no meta- analyses or other comprehensive 
reviews. Instead, these points are supported by only one journal article (in Econometrica) and 
two reports (from the Wallace Foundation and McREL) that bear only on the claim’s leadership 
aspect— not the teacher aspect— and a self- published document by value-added proponent 
William Sanders.'’ 

The summary and the research dted are overvdielmingly grounded in the assumption that 
student test scores arB the only important outcome of interest. This presumption is built into 
GTGL by repeated, confident references to claims of “achievement” gains that would more 
accurately be called “test-score” gains. Euphemisms embodying this assumption include 
“improving academic outcomes,” “greatly improve student achievement,” “recognize and reward 
effectiveness,” “successful at increasing student learning,” and “take performance into account.” 
This is not to say that standardized test scores have no bearing on judging quality, but the 
validity of GTGL’s argument would be stronger if clear and limited language were used and if 
other recognized points of view and outcome measures were acknovdedged. 

Similar problems underlie the prescription of performance- pay and incentive- pay for providing 
and retaining well- qualified teachers for challenging schools (or shortage areas) . The reliance on 
pay-for-performance is most stark in the “Teacher and Leader Innovation Fimd” section of 
GTGL. Teacher motivation is assumed to be powerfully economic, as though no other 
motivations aro relevant. This assumption is particularly faulty with respect to human service 
professions such as teaching, since entering many of these professions is a choice to abjure the 
unbridled path to economic gain in favor of nurturing other life and career values. 



V. Review of the Report’s Methods 

As noted, GTGL looks at only a limited spectrum of the relevant existing research and theory in 
drawing its conclusions and does not identify or explain this narrow perspective. In judging 
school quahty through standardized testing, for instance, GTGL implicitly adopts one school of 
thought without any suggestion that there are alternative criteria for evaluation^ or mention of 
the widespread criticisms of the high- stakes testing approach.*^ 

Further, the GTGL research summary never dtes or considers the extensive body of research 
concerning the specific disciplinary knovdedge of teacher education— the sorts of knovdedge 
insights developed by practitioners of any profession. There is not a sin^e reference to key 
journals in the field such as The Journal ofTeaeher Edueation, where such unique professional 
knovdedge might be revealed (see the discussion of citations later in this review). 

Similarly, GTGL asserts that meeting the “highly qualified teacher” criterion of NCLB does not 
predict that teachers will be successful at improving student learning. This may be true, but the 
research review is an incomplete and skewed survey of the available evidence and thus misleads 
readers. For example, it omits any discussion of the variability of quality in the teacher 
education programs that lead to teachers being designated “highly qualified.” Standards such as 
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a bachelor’s degree and state certification are vague and uneven. State licensing and regional 
accreditation vary widely. 

Voluntary professional accreditation is divided between two organizations,^ and such 
accreditation is not mandatory in many states.'® These accreditation issues are not mentioned in 
the report -- a major omission. This variation is also illustrated by the fact that in C a l if ornia 
approximately 20% of teacher preparation is done by two private universities. Chapman 

The GTGL research summary never cites or considers the extensive body 
of research concerning the specific disciplinary knowledge of teacher 
education. 

University and National University, through their transported (off-campus) programs," 
operating throughout the state without professional accreditation and with, at best, a modicum 
of full-time, doctorally prepared faculty.'^ All these variables are swept into a sin^e term: 

“highly qualified.” This designation— with its admitted lack of predictive ability— is then used as 
the basis for the Blueprint’s policy choices. 

The same problem arises with the research summary’s assertion that, except for mathematics 
teachers with content degrees, there is “no evidence that teachers with master’s degrees perform 
better.” Such a sweeping claim has little practical meaning or value. To be useful for detailed 
policymaking, the research dted would need to control for the nature of the master’s degree 
programs, but no such research is provided. This omission is highlighted by GTGL itself 
acknowledging that teachers with secondary mathematics master’s degrees do “perform better.” 
Whether institutions accredited by NCATE (the National Goundl for Accreditation of Teacher 
Education) offered the graduate programs, for example, would be germane. Whether the 
programs were related to the acquisition of a new license or endorsement would also be 
important, as would vdiether degrees were awarded by off-campus programs with a one- 
professor instructional scheme . GTGL ignores all these types of variability in certification and 
graduate teacher education. 

The GTGL research summary also misleads through the selective and narrow choice of research 
dted about the relevance and selectivity of teacher- education programs, pointing to the 
relatively low SAT scores of teacher candidates. SAT scores are not the sine qua non of 
selectivity. Studies in the Journo/ of Teacher Education and elsewhere have shown teacher 
education students outperform other students in their general education coursework.'"' Research 
has also shown that teacher test scores are a poor predictor of “value-added” teacher 
effectiveness.'^ This finding is presented, among other places, in the condusion from a peer- 
reviewed meta- analysis: “After reviewing a rather large body of validity evidence, we discovered 
that test scores have been less related to teaching performance than [teacher education] 
students’ success levels in the preservice programs the tests were designed to hold 
accountable.”'® Finally, for well over two decades NCATE- accredited programs have imposed 
restrictive admission requirements. At such institutions, teacher education has long since lost 
its status as a major of last resort. 
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GTGL celebrates as emerging innovations “promising models” that are basically the same field 
and dinical experiences that have been required in NCATE- accredited programs since the 
1970s. These include focusing on work in classrooms, capstone projects, and portfolios. The 
report lacks a basic awareness of the landscape of teacher education, including the wide 
variability among the states in hcensure standards and adherence to professional accreditation. 
The reforms promoted by reports such as GTGL— such as greater selectivity in admissions and a 
strong component of clinical and field experiences— are well-established practices at hundreds 
of colleges and universities. When we look at over 650 NCATE- accredited institutions, which 
taken together are a good representation of mainstream teacher education, we get a view of 
teacher education that is more selective and more likely to use the practices and approaches 
favored by the report. Unless we make such groupings, the generalized approach of GTGL leads 
us to ne^ect the established successes on vdiich reform can be built. The research summaiy 
misses the trees for the forest. 



VI. The Report’s Use of the Research Literature 

The GTGL research summary is not well grounded in the literature of teacher education. To 
illustrate, in 2005 as the result of an extensive deliberative process, the American Education 
Research Association published Studying Teacher Education/^ edited by eminent scholars 
Marilyn Cochran- Smith and Keimeth Zeichner. In 2006, the American Association of Colleges 
for Teacher Education (AACTE, another prominent organization) awarded Studi/mg Teacher 
Education its Outstanding Publication Award, describing it as a work “that will serve as a guide 
to the profession for years to come.”^*’ This volume is not dted in GTGL. Nor are Cochran- Smith, 
Zeichner, or the 14 other panel members vdio contributed to writing that seminal book. 

The Journal of Teacher Education, a publication of the AACTE and arguably the most 
prestigious journal pertaining to teacher education, keeps current a list of “The 50 Most- 
Frequently Read Articles. About 100 authors are represented among these articles. In GTGL, 
the work of just one of these authors is dted. The Journal of Teacher Education also publishes a 
list of “The Most-Frequently Cited Articles. Approximately 70 authors are represented in 
these articles. Again, publications from only one are dted in GTGL. In both cases, the person 
dted was the same— the work of Linda Darling- Hammond, a respected scholar who played a 
prominent role in the Obama presidential campaign. 

Of the 80 or so sources used for the report, approximately 10% are drawn from peer-reviewed 
journals. Although some of the other references are worthy of dtation, GTGL’s overall sampling 
of the research base is inadequate. 



VII. Usefulness of the Report for Guidance of Policy and Practice 

Since A Nation at Risk, we have become accustomed to the federal government repeating the 
errors propagated by advocacy think tanks and publishing reports that present themselves as 
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science but v\bich lack the essential characteristics of research or scholarship. Such reports blur 
important categories, cherry- pick from a narrowband of publications, and ignore much of the 
mainstream practice and research in the field. Great Teachers and Great Leaders is 
unfortunately part of this genre. It offers itself as a targeted review of teacher- education 
knovdedge vdien it is in fact a partisan political text that starts with a conclusion and then finds 
evidence to support it. 
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